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Abstract 

Purpose Much collective wisdom and experience have been 
gained as an increasing number of Life Cycle Assessment 
(LCA) reviews are conducted. However, specifics on how 
and when to conduct critical review of LCA studies are still 
lacking. Toward this need, a technical session entitled “LCA 
Critical Review” was held during the Life Cycle Management 
(LCM) conference in Gothenburg, Sweden, 26 August 2013. 
The goal of the session was to have experts address LCA 
critical review as well as engage attendees in discussing gaps 
in the current guidance and how the review process can be 
improved. 

Methods The LCM session consisted of six presentations 
followed by open discussion with all session attendees. This 
paper begins with a review of the current state-of-the-practice 
in LCA critical review (CR) followed by a summary of the 
LCM session. It concludes with suggestions for how the 
newly drafted technical specification, ISO TS 14071 Critical 
review processes and reviewer competencies, can be im¬ 
proved as it is being developed. 

Results and discussion ISO TS 14071 promises to provide 
additional guidance to move the practice forward. But at only 
eight pages in length, its potential effectiveness appears mod¬ 
erate. Additional detailed guidance is needed to move the 
critical review process toward increased uniformity and clarity 
of practice, for example, when critical review is necessary. 
Conclusions A session on LCA critical review is planned to 
be held during the Life Cycle Management 2015 conference 
which will occur in Bordeaux, France (http://lcm2015.org/). 
Discussion on these issues related to LCA review will be 
continued. 
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1 Introduction 

Critical review has existed almost as long as life cycle 
assessment (LCA) itself. The intent of reviewing the con¬ 
duct and reporting of an LCA is to further improve the 
study, remove serious errors, and deter deceptive practice, 
especially in the case of comparative studies. Although 
“peer” review for LCA studies was first proposed in the 
1993 SETAC guidelines “A Code of Practice” (Consoli et 
al 1993), the first detailed guidance for critical review of 
LCA was not published until 1997 (Weidema 1997; 
Klopffer 1997). The international standards series for 
LCA, originally developed between 1996 and 2000, and 
since updated to ISO 14040 and ISO 14044 in 2006, has 
addressed the issue and makes review a requirement for 
an LCA study. ISO 14040 describes, among other things, 
three types of “critical review” which are optional in 
general, but mandatory “for LCA studies used to make a 
comparative assertion disclosed to the public.” 
Additionally, since 2006, the minimum number of experts 
in a “review by interested parties” is now three (including 
the chair). 

Much collective wisdom and experience have been gained 
with the conduct of reviews in practice. However, specifics on 
how and when to conduct critical review of LCA studies are 
still lacking. This has led to the development of additional 
guidance under ISO TS 14071. 1 The intent of this proposed 
international technical specification (expected completion in 


1 ISO/PDTS 14071 Environmental management—Life cycle assess¬ 
ment—Critical review processes and reviewer competencies: Additional 
requirements and guidelines to ISO 14044:2006b, technical specification 
under development; 2013-09-2. 
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2014) is to provide requirements and guidelines for 
conducting a critical review and the competencies required 
(Finkbeiner 2013). 

This paper begins with a brief summary of the current 
state-of-the-practice in LCA review, as directed by the ISO 
14044 standard. Next, a summary is given based on six 
presentations and open discussion from a technical session 
entitled “LCA Critical Review” which was held during the 
Life Cycle Management conference in Gothenburg, 
Sweden, 26 August 2013. The goal is to identify existing 
gaps in guidance and weaknesses in how reviews are 
planned and conducted in order to propose how the prac¬ 
tice can be strengthened. Suggestions are offered on how 
to improve the ISO 14071 Technical Specification, on 
critical review processes and reviewer competencies, as it 
is being developed. 


2 ISO-Defined LCA review 

First, we need to address the terms “critical review” and 
“verification” which tend to be used interchangeably, even 
though they are philosophically different (Grahl and 
Schmincke 2012). “Critical review”, like scientific peer re¬ 
view, checks the scientific capacity of a study and the appro¬ 
priateness of methods and assumptions. In comparison, “ver¬ 
ification” is an audit exercise and is the validation of confor¬ 
mance' against specified requirements, based on an evidence- 
based approach supported by principles of objectivity and 
repeatability (see also ISO 19011:2011). Unlike verification, 
critical review does not rely on objective evidence, but instead 
relies on expert judgment about LCA methodology and prac¬ 
tice. The well-known five criteria presented by ISO 14044 
clause 6.1 (see box) for a critical review are not 
operationalizable in the same way verifiable requirements are. 


ISO 14044 (2006b) specifically identifies five requirements in the critical 
review process 
Clause 6.1 General 

The critical review process shall ensure that: 

- the methods used to carry out the LCA are consistent with this 
international standard; 

- the methods used to carry out the LCA are scientifically and technically 

valid; 

- the data used are appropriate and reasonable in relation to the goal of the 

study; 

- the interpretations reflect the limitations identified and the goal of the 
study; and 

- the study report is transparent and consistent. 


2 The word “conformance” is preferred over “compliance” which implies 
agreement with some external authority. 


ISO 14044 (2006b) defines “critical review” as the “pro¬ 
cess intended to ensure consistency between a life cycle 
assessment and the principles and requirements of the 
International Standards on life cycle assessment. 3 ” Since the 
ISO standard specifies “critical review,” this terminology is 
used in this paper even though the intent of current LCA 
reviews may be more in line with a checklist-type approach 
to verify confonnance with the ISO standards. 

2.1 Conduct of an LCA review 

The ISO 14044 standard requires that the scope of the study 
defines whether a critical review is necessary and, if so, how to 
conduct it, the type of critical review needed, and who would 
conduct the review, and their level of expertise. The standard 
is very specific when it comes to review of results that are 
intended to be used to support a comparative assertion 
intended to be disclosed to the public. In order to decrease 
the likelihood of misunderstandings or negative effects on 
external interested parties, a panel of interested parties shall 
conduct critical reviews on these types of studies. 

Clause 6.2 of ISO 14044 indicates that a critical review 
may be carried out by a single internal or external expert. In 
either case, the expert must be independent of the LCA. Under 
clause 6.3, critical review may also be carried out by a panel of 
interested parties. In such a case, an external independent 
expert should be selected by the original study commissioner 
to act as chairperson of a review panel of at least three 
members. Based on the goal and scope of the study, the 
chairperson should select other independent qualified re¬ 
viewers. This panel can include other interested parties who 
are affected by conclusions drawn from the LCA, such as 
government agencies, non-governmental groups, industry 
competitors, and affected industries. 

Typically in practice, members of the review team (also 
known as the panel) bring a combined expertise that offers 
complete coverage of all aspects involved in conducting an 
LCA. At a minimum, these specialties must include knowl¬ 
edge in the following areas: 

• Conformity to the ISO standard at goal and scope 
definition; 

• Data/inventory (LCI) relevant to the industry sector(s) 
being studied; 

• Life cycle impact assessment (LCIA) methods, data and 
modeling of relevant impact categories; and 

• Overall flow and management of the review process. 


3 The definition in ISO 14044 for critical review (section 3.45) includes 
NOTE 1 The principles are described in ISO 14040:2006a, 4.1 and 
NOTE 2 The requirements are described in this International Standard. 
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While each member of the panel reads the whole report, the 
depth of the critical reading may and should differ. 
Sometimes, one member of the review team is a technical 
specialist for the product, industry, or production technology 
which is in the focus of the study; this member is not neces¬ 
sarily an LCA expert, although it is usually better that all 
members are familiar with the approach (Klopffer 2005, 
2012 ). 

The expert review of LCAs requires a close (as in intimate) 
and open cooperation between the commissioner of the study, 
the practitioner, and the reviewer or the review team. All 
identities are known, unlike the peer review of scientific 
journals, which are often conducted in a single blind approach 
(the reviewers are anonymous). LCA critical review is also 
much more time consuming. 

The review statement and review panel report, as well as 
comments of the expert and any responses to recommenda¬ 
tions made by the reviewer or by the panel, are to be included 
in the LCA report. 

2.2 Review of data 

An important aspect concerning quality and credibility of any 
LCA is access to the underlying inventory data. In fact, the 
confidentiality of sensitive data is the most important reason 
for conducting a professional critical review. ISO 14044 does 
not directly describe data review; however, draft ISO TS 
14071 indicates that the critical review should cover all as¬ 
pects of an LCA, including data and calculation procedures: 11 
The critical review should cover all aspects of an LCA, 
including data, calculation procedures for linking the unit 
processes into product systems, life cycle inventory impact 
assessment methodologies, characterization factors, and cal¬ 
culated LCI and LCIA results.” 

However, since not all data are publicly available, or they 
are published in a highly aggregated form, reviewers have to 
judge data quality and appropriateness, for example, by ran¬ 
dom sampling or tracking of calculations. In order to do so, 
they have to have access to the data, and the commissioner and 
practitioner must provide this access. This may be complicat¬ 
ed if third parties provide data or are not prepared to reveal 
confidential issues. Subcontractors have to be included in data 
transparency for the review team or at least for one member of 
the panel responsible for data quality. 

2.3 Timing 

Critical review can be conducted at various stages of comple¬ 
tion of the LCA study. ISO 14044 does not elaborate on this 
issue. However, draft ISO TS 14071 offers guidance if the 
review is performed concurrently or at the end of the study 
(i.e., post-study): 


- If the critical review is performed at the end of the LCA 
study, the process starts when the draft LCA report is 
provided to the reviewers). 

- If the critical review is perfonned concurrently with the 
study, the process starts as early as the study commission¬ 
er and the practitioner decide. The various milestones at 
which the reviewer(s) may be asked to submit comments 
and recommendations are: 

1. The goal and scope definition; 

2. Inventory analysis, including data collection and 
modelling; 

3. Impact assessment; 

4. Life cycle interpretation; and 

5. Draft LCA report. 

The process choice does not change the deliverables of the 
critical review process. In all cases, the critical review process 
shall clearly define and document its assessment of choices 
made by the study practitioners in areas including boundaries, 
functional unit, data, allocations, indicators, and so on. 


3 “LCA Critical Review” session during the Life Cycle 
Management 2013 Conference in Gothenburg, Sweden 

During the Life Cycle Management conference held in 
Gothenburg, Sweden, a session entitled “LCA Critical 
Review” was held on 26 August 2013. The stated aim of the 
session was to facilitate discussion on the lack of a standard¬ 
ized review process, as well as the role of practitioner certifi¬ 
cation, the conduct of quality reviews of LCA datasets and 
databases contained in publicly available reports and journal 
articles, and how to approach reviewing LCA methodology 
and applications. The coauthors and titles of the six presenta¬ 
tions are shown in Table 1. The presenters’ names are 
underlined. The conference proceedings are available at 
http://conferences.chalmers.se/index.php/LCM/LCM2013/ 
schedConf/presentations. 

The session was highly successful in meeting the intended 
goal. The speakers were excellent and informative; the room 
was packed to overflowing, and the questions and discussion 
that were generated were high caliber. From the questions and 
discussion, it was apparent that numerous people in the room 
had experience serving on critical review panels. 

3.1 Key messages from the LCA Critical Review Session, 26 
August 2013 

Following is a discussion of the key messages, a combination 
of reflections and questions, raised by the presenters during 
their presentations and by attendees during the subsequent 
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Table 1 Session at the LCM 2013 Conference: “LCA Critical Review” 

Session chairs: Mary Ann Curran, Steven B. Young 

The Critical Review according to ISO 14040+44—How and why it came 
about. 

Walter Klopffer 

LCA Consult & Review, Germany 

International survey on critical review and verification practices in LCA 
with a focus in the construction sector 

Sebastien Lasvaux 1 , Yann Leroy 2 , Capucine Briquet 2 , Jacques 
Chevalier 1 

'Universite Paris-Est, Centre Scientifique et Technique du Batiment 
(CSTB), France; 2 Laboratoire Genie Industriel—Ecole Centrale 
Paris, France 

Guidance for conducting collaborative critical peer review using co¬ 
design tools and reviewer profiles 

Stephane MOREL 1 ’ 2 , Philippe OSSET 3 , Sarah ERTEL 4 , Franck 
AGGERI 2 

Renault SAS, France; 2 Mines Paristech, CGS, Paris, France; 
3 Solinnen, Paris, France; 4 Be-linked, Business & Community 
Intelligence, Paris, France 

Radically reducing the costs of panel critical reviews according to ISO 
14040 

Bo Pedersen Weidema 1 ' 3 , Kim Christiansen 2 , Gregor Wemet 3 
'Aalborg University, Demnark; 2 Danish Standards, Denmark; 3 The 
ecoinvent Centre, Switzerland 

Critical review of LCA—Essential for quality and understanding 
Matthias Schulz , Ivo Mersiowsky 
DEKRA Consulting GmbH, Germany 

LCA critical review experiences and suggestions for improvement 
Fredy Dinkel, Cornelia Stettler, Emil Franov 
Carbotech AG, Switzerland 


open discussion period. Three themes are presented: The 
Reviewers, The Review Process, and Managing the 
Outcome of the Review. 

3.1.1 The reviewers 

The first question raised was how many reviewers are needed 
to conduct a review? Three are required by ISO, but are two 
sufficient, especially in the case of small companies with 
limited budgets? Then, the question becomes, where do we 
find (qualified) reviewers? Reference was made to the “usual 
LCA reviewers” meaning that a small pool of reviewers is 
being called on repeatedly. In addition to overtaxing this 
limited pool, interested parties may not always be included. 
Stephane Morel of Renault presented the idea of soliciting the 
participation of a large review panel that includes non-LCA 
expert stakeholders. The concept extends roles in the conven¬ 
tional critical review to include an “enabler,” acting as a 
facilitator who brings different perspectives together, and 
“observers” who provide a broader perspective to the group 
and provide more global context for the study. 


3.1.2 The review process 

Attendees recognized that it is common, and acceptable, for 
the review process to vary depending on the type of product 
being reviewed: reports, data, databases, and so on. There are 
also concurrent versus post-study reviews, although post¬ 
study reviews are more commonly done. Therefore, the num¬ 
ber of iterations within the review cycle (i.e., repeating the 
review process as revisions are made) varies. The need for 
consistent criteria to guide different reviews was also clear. If 
publicly available, the review guidelines developed by 
DEKRA (presented by Schulz) could provide insight. 

The role of reviewers Conventionally, the LCA critical review 
panel engages “interactively” (Kldpffer) as part of the triangle 
with the study commissioner and analysts, as described in ISO 
14044. However, this triangle could be augmented, as de¬ 
scribed by Morel to add non-LCA experts and an “enabler” 
whose role would be to facilitate the expanded group. 

Dinkel pointed out that the duty of the reviewer is not only 
to do an “administrative ISO check,” but to also be a discus¬ 
sion partner accompanying the LCA project. In general, 
forming fonnal partnerships with reviewers was viewed as a 
potentially good idea, but it was recognized that the reviewer 
as partner is not however a coauthor of the LCA study. As 
emphasized by Weidema and also highlighted by the follow¬ 
ing discussion, the panel is obliged to maintain a level of 
neutrality toward the LCA study. There is, therefore, a poten¬ 
tial tension that develops between the panel’s objectivity 
versus its involvement as participatory partner in a study. 
The concern was that reviewers would slip into an analytical 
role and becomes vested in a project, thus risking their 
independence. 

Quality of the review Lasvaux presented survey results 
from LCA experts that showed variability in the detail 
and quality of critical review activity. One highlight was 
that whereas review of methods and assumptions is 
usually performed comprehensively, the examination of 
data and therefore quantitative results is often limited. 
This may be a function of the critical review panel’s 
mandate, due to limitations of time, or as a result of 
weak transparency or poor accessibility to data sets. 
Schulz outlined review guidance that included a system¬ 
atic checklist to ensure comprehensiveness and quality. 

Weidema proposed steps to drastically reduce the cost of 
critical review without compromising thorouglmess and over¬ 
all quality. This procedure has three elements: 

1. A fixed panel for all reviews—Each commissioner (e.g., a 
specific company or agency) could establish a fixed re¬ 
view panel (i.e., a pool) which would reduce 
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administrative efforts for composing, communicating 
with, and remunerating the review panels 

2. An already critically reviewed background database—The 
most important element, is based on the 1:1 relationship 
between the number of LCAs and the number of unit 
processes in an LCA database, implying that every time 
you create a new unit process, you also create a new LCA 
(for the reference product of that unit process). In a situation 
where critical reviews have already been performed for 
each datasets that make up the complete global database, 
as well as for each algorithms that the database use for 
calculating the LCA results, then it would only be neces¬ 
sary to review the new “foreground” dataset or algorithms 
that you add for the decision context in which it is used. By 
limiting the critical review to the newly added data and 
algorithms, the costs are obviously radically reduced, while 
ensuring that every single dataset is critically reviewed 
enhances the overall quality of the review will be enhanced 
compared to the present-day situation. 

3. A software-supported review procedure—Performing the 
reviews purely electronically in order to reduce travel 
costs and meeting time. This approach would have the 
added benefit of reducing environmental impacts of the 
review itself. 


3.1.3 Managing the outcome of the review 

Critical review requirement and outcomes Part of the discus¬ 
sion covered “When is critical review actually necessary?” 
Klopffer had reviewed the ISO 14040/44 requirements, but 
the members of the audience suggested that the definition of 
“comparison” and “comparative assertion” are still confusing 
and open to debate. 4 The question of LCA used in 
Environmental Product Declarations (EPDs) was one exam¬ 
ple; even though a comparative assertion is not explicitly 
stated in a report, comparison is the aim of giving users data 
and infonnation in an EPD. 

A somewhat related discussion concerned the outcome of a 
critical review process. In particular, how can disagreements 
between parties be handled? What form of enforcement can be 
applied, if any? Furthermore, Klopffer emphasized that, given 
that there are no single “correct” findings in LCA, 5 the resulting 
report cannot be “certified,” although this is a common notion 
(including one presented by BMW the next day at the same 
conference). The report fonn and “assurance” produced by the 
critical review is an area of valuable future attention. One 

4 Two camps clearly, and quickly, emerged on what constitutes compar¬ 
ative assertion, whether it is must be stated in the LCA report or if a 
reader-conducted comparison is sufficient to trigger the review 
requirement. 

5 That is, results depend on several factors, including which modeling 
assumptions, scope, and data are used. 


direction is to move towards more detailed, sector-specific guid¬ 
ance for perfonning LCA studies, which might be a basis for 
auditable criteria as reference to future reviews. 


4 Discussion 

In general, the presentations and discussion during the LCM 
session highlighted the need for (1) increasing the quality of 
both people involved and data in LCA; (2) increasing the 
speed and efficiency; and (3) reducing the cost of critical 
review. There was some convergence of opinion around areas 
like the need for quality and rigor. However, there was robust 
debate on areas such as when critical review is necessary— 
particularly around what constitutes “comparative asser¬ 
tion”—and what role reviewers need to play. The key issues 
raised during the LCM session are mapped against the sec¬ 
tions in the draft ISO TS 14071 where they are addressed, 
along with an indication on how well the new technical 
specification provides guidance (Table 2). 

5 Conclusions 

As mentioned in the Introduction, much collective wisdom 
and experience have been gained as an increasing number of 
LCA reviews are conducted. In addition, the draft ISO 14071 
promises to provide additional guidance to move the practice 
forward. But at only eight pages in length, the effectiveness of 
the new technical specification appears moderate. Additional 
detailed guidance is needed if it is to move the critical review 
process toward better uniformity. Some issues identified here 
may be outside the scope of the technical specification, such 
as enforcement. In that case, an alternate mechanism for 
providing guidance for these areas is needed. 

A session on LCA critical review is planned to be held 
during the Life Cycle Management 2015 conference which 
will occur in Bordeaux, France. Details about the conference 
can be found at http://lcm2015.org/. Discussion on these 
issues related to LCA review can be continued then. 
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